Automatic Replicant Backup over USB using rsync

I have been using Replicant on the Samsung SIII I9300 for over two years. I have written before on taking a backup of the phone using rsync but recently I automated my setup as described below. This work was prompted by a screen accident with my phone that caused it to die, and I noticed that I hadn’t taken regular backups. I did not lose any data this time, since typically all content I create on the device is immediately synchronized to my clouds. Photos are uploaded by the ownCloud app, SMS Backup+ saves SMS and call logs to my IMAP server, and I use DAVDroid for synchronizing contacts, calendar and task lists with my instance of ownCloud. Still, I strongly believe in regular backups of everything, so it was time to automate this.

For my use-case, taking backups of the phone whenever I connect it to one of my laptops is sufficient. I typically connect it to my laptops for charging at least every other day. My laptops are all running Debian, but this should be applicable to most modern GNU/Linux system. This is not Replicant-specific, although you need a rooted phone. I thought that automating this would be simple, but I got to learn the ins and outs of systemd and udev in the process and this ended up taking the better part of an evening.

I started out adding an udev rule and a small script, thinking I could invoke the backup process from the udev rule. However rsync would magically die after running a few seconds. After an embarrassing long debugging session, finally I found someone with a similar problem which led me to a nice writeup on the topic of running long-running services on udev events. I created a file /etc/udev/rules.d/99-android-backup.rules with the following content:

ACTION=="add", SUBSYSTEMS=="usb", ENV{ID_SERIAL_SHORT}=="323048a5ae82918b", TAG+="systemd", ENV{SYSTEMD_WANTS}+="android-backup@$env{ID_SERIAL_SHORT}.service"
ACTION=="add", SUBSYSTEMS=="usb", ENV{ID_SERIAL_SHORT}=="4df9e09c25e75f63", TAG+="systemd", ENV{SYSTEMD_WANTS}+="android-backup@$env{ID_SERIAL_SHORT}.service"

The serial numbers correspond to the device serial numbers of the two devices I wish to backup. The adb devices command will print them for you, and you need to replace my values with the values from your phones. Next I created a systemd service to describe a oneshot service. The file /etc/systemd/system/android-backup@.service have the following content:

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/android-backup %I

The at-sign (“@”) in the service filename signal that this is a service that takes a parameter. I’m not enough of an udev/systemd person to explain these two files using the proper terminology, but at least you can pattern-match and follow the basic idea of them: the udev rule matches the devices that I’m interested in (I don’t want this to happen to all random Android devices I attach, hence matching against known serial numbers), and it causes a systemd service with a parameter to be started. The systemd service file describe the script to run, and passes on the parameter.

Now for the juicy part, the script. I have /usr/local/sbin/android-backup with the following content.

#!/bin/bash

DIRBASE=/var/backups/android
export ANDROID_SERIAL="$1"

exec 2>&1 | logger

if ! test -d "$DIRBASE-$ANDROID_SERIAL"; then
    echo "could not find directory: $DIRBASE-$ANDROID_SERIAL"
    exit 1
fi

set -x

adb wait-for-device
adb root
adb wait-for-device
adb shell printf "address 127.0.0.1\nuid = root\ngid = root\n[root]\n\tpath = /\n" \> /mnt/secure/rsyncd.conf
adb shell rsync --daemon --no-detach --config=/mnt/secure/rsyncd.conf &
adb forward tcp:6010 tcp:873
sleep 2
rsync -av --delete --exclude /dev --exclude /acct --exclude /sys --exclude /proc rsync://localhost:6010/root/ $DIRBASE-$ANDROID_SERIAL/
: rc $?
adb forward --remove tcp:6010
adb shell rm -f /mnt/secure/rsyncd.conf

This script warrant more detailed explanation. Backups are placed under, e.g., /var/backups/android-323048a5ae82918b/ for later off-site backup (you do backup your laptop, right?). You have to manually create this directory, as a safety catch to not wildly rsync data into non-existing directories. The script logs everything using syslog, so run a tail -F /var/log/syslog& when setting this up. You may want to reduce verbosity of rsync if you prefer (replace rsync -av with rsync -a). The script runs adb wait-for-device which you rightly guessed will wait for the device to settle. Next adb root is invoked to get root on the device (reading all files from the system naturally requires root). It takes some time to switch, so another wait-for-device call is needed. Next a small rsyncd configuration file is created in /mnt/secure/rsyncd.conf on the phone. The file tells rsync do listen on localhost, run as root, and use / as the path. By default, rsyncd is read-only so the host will not be able to upload any data over rsync, just read data out. Next rsync is started on the phone. The adb forward command forwards port 6010 on the laptop to port 873 on the phone (873 is the default rsyncd port). Unfortunately, setting up the TCP forward appears to take some time, and adb wait-for-device will not wait for that to complete, hence an ugly sleep 2 at this point. Next is the rsync invocation itself, which just pulls in everything from the phone to the laptop, excluding some usual suspects. The somewhat cryptic : rc $? merely logs the exit code of the rsync process into syslog. Finally we clean up the TCP forward and remove the rsyncd.conf file that was temporarily created.

This setup appears stable to me. I can plug in a phone and a backup will be taken. I can even plug in both my devices at the same time, and they will run at the same time. If I unplug a device, the script or rsync will error out and systemd cleans up.

If anyone has ideas on how to avoid the ugly temporary rsyncd.conf file or the ugly sleep 2, I’m interested. It would also be nice to not have to do the ‘adb root’ dance, and instead have the phone start the rsync daemon when connecting to my laptop somehow. TCP forwarding might be troublesome on a multi-user system, but my laptops aren’t. Killing rsync on the phone is probably a good idea too. If you have ideas on how to fix any of this, other feedback, or questions, please let me know!

9 Replies to “Automatic Replicant Backup over USB using rsync”

  1. Where did you get rsync for Android from? Does it come with Replicant ROM? Wait… find | grep rsync. There it is: ./system/xbin/rsync! Thank you for being my rubber duck.

    PS. I am running CM12.1

    • You are welcome 🙂 It has always been pre-installed on the android ROM’s that I’ve worked with.

      /Simon

  2. Doesn’t that rsync configuration allow all applications on the device to access (and modify!) any file on the device? They only need to open a TCP connection to localhost:873 and speak the rsync protocol.

    • Good point! Thanks. I only run apps for which I have source code on my devices, so I have some trust in that nothing I would run would do this. Resolving this is part of the open issues though. Using the ssh-protected adb transport seems like the best idea.

      Maybe it is possible to use the rsync -e parameter somehow. For example: rsync -e ‘adb shell /bin/sh’ or something? Quick experimentation didn’t work out, but I’m sure it is possible to do something here.

      /Simon

    • Small clarification: modification of files wouldn’t be possible, since the rsync server is read-only.

      If malware hits my phone, I doubt that this would be the most popular attack vector anyway (just use any unfixed android security problem in replicant).

      Still, it is a weakness and should be fixed.

      /Simon

  3. Great idea! I already had a pair of scripts which utilized adb pull/push to do backups (see [1]). But I love the idea of using rsync for it, since it speeds up incremental updates *a lot*.
    I’ve had another go at it combining ideas from your post with ideas from pts[2]. I had to get a little creative since certain versions of Android don’t have rsync (e.g. OxygenOS), don’t allow “adb root” (e.g. CyanogenOS) or don’t allow you to set timestamps (it’ll print “mkstemp “…” (in root) failed: Operation not permitted (1)” errors) … I documented my results on my blog[3].

    [1] http://blog.bilak.info/2015/05/01/android-backup-and-restore-with-adb/
    [2] http://ptspts.blogspot.de/2015/03/how-to-use-rsync-over-adb-on-android.html
    [3] http://blog.bilak.info/2016/01/29/backup-and-restore-your-android-phone-with-adb-and-rsync/

  4. Thanks for the great page. It was quite helpful.

    I’ve modified the script to add some of the wishlist items. In particular, it restricts rsyncd access to localhost, created a pid file for rsyncd, kills rsyncd after completion. It also adds a randomly generated “shared secret” for accessing the rsyncd so other apps presumably wouldn’t know the secret, and sets permissions on the config files to 600.

    FYI I run this script by hand without the automatic start at plugin of phone. It works great. Just remember to pass in the phone’s device serial on the command-line.

    Here is the modified script:

    #!/bin/bash

    BASEDIR=/var/backups
    PREFIX=$BASEDIR/android
    export ANDROID_SERIAL=”$1″

    # generate random secret
    secret=$(dd if=/dev/urandom bs=1 count=32 2>/dev/null | base64 -w 0 | rev | cut -b 2- | rev)

    #log to logger
    exec 2>&1 | logger

    #log to logfile
    #exec >$BASEDIR/log/android_backup.`date +%Y%m%d-%H%M%S`.log 2>&1
    #exec >>$BASEDIR/log/android_backup.log 2>&1

    if ! test -d “$PREFIX-$ANDROID_SERIAL”; then
    echo “could not find directory: $PREFIX-$ANDROID_SERIAL”
    exit 1
    fi

    set -x

    adb wait-for-device
    adb root
    adb wait-for-device
    adb shell “umask 077 && printf ‘backup:$secret\n’ > /mnt/secure/rsyncd.secrets”
    adb shell “umask 077 && printf ‘address = 127.0.0.1\npid file = /mnt/secure/rsyncd.pid\nuid = root\ngid = root\n[root]\n\tpath = /\n\thosts allow = 127.0.0.1\n\tauth users = backup\n\tsecrets file = /mnt/secure/rsyncd.secrets\n’ > /mnt/secure/rsyncd.conf”
    adb shell “umask 077 && rsync –daemon –no-detach –log-file=/storage/emulated/0/rsyncd/rsyncd.log –config=/mnt/secure/rsyncd.conf” &
    adb forward tcp:6010 tcp:873
    sleep 2
    RSYNC_PASSWORD=$secret rsync -av –delete –exclude /dev –exclude /acct –exclude /sys –exclude /proc rsync://backup@localhost:6010/root/ $PREFIX-$ANDROID_SERIAL/
    : rc $?
    adb forward –remove tcp:6010

    # kill rsyncd
    adb shell ‘kill `cat /mnt/secure/rsyncd.pid`’

    # remove temporary files
    adb shell rm -f /mnt/secure/rsyncd.conf /mnt/secure/rsyncd.pid /mnt/secure/rsyncd.secrets

    • It is also worth mentioning that on my nexus5x running cyanogenmod, the above script backed up my main internal storage 3 times. I had to add:
      –exclude /mnt –exclude /storage/emulated
      to the rsync command in order to prevent this. With these two excluded, it leaves /data/media/0 as the place where most of your stuff ends up.